The present invention relates to systems and methods for the evaluation and modeling of computer system engines, and in particular to methods and systems for providing in an offline environment, without the need to physically test a particular computer system (or a number of systems), information relative to the processing capacity and/or performance of such system(s), such information being derived by analysis of the components of the computer system.
Users of computer systems, as well as information services administrators and others who may be responsible for evaluating, purchasing, or implementing computer systems, e.g., for an enterprise, often confront multiple choices as to a large plurality of possible system configurations. For instance, within the context of an enterprise server computer system, a company considering the purchase of such a system will have many options as to varying combinations of “candidate” systems having respective system components of differing architecture and performance traits.
Some of these candidate systems may be newly-manufactured systems, while some candidate systems may represent combinations of already-installed components owned by the company, together with additional new upgrade components. Enterprise servers may comprise, among other things, multiple central processing units (CPUs), which may in turn be available in a variety of processor clock speeds (proportional to the number of processing operations such a processor can perform in a second).
The processor(s) may also be configured in a variety of architecture classes or bit, architectures, which partly determine the “channel width” of a given processor. For instance, current Pentium-class processor chips typically employ a 32 bit architecture, whereas currently-used Unix servers may more commonly employ 64 bit architecture classes. For the purposes of this invention, a computer system may be considered as comprising a computer processor or any combination of operationally linked co-operating computer processors, whether located within a single computer unit, or distributed remotely.
Numerous other component variables may contribute to the operation of a particular system, and the ability to mix and match components of varying performance traits or function (e.g., the ability to vary the amount of Random Access Memory in the system, or to choose from among a variety of differing data buses, coprocessors, and special function components) further complicates the task of determining and comparing overall system performance as between custom-configured, or “off-the-shelf,” computer systems that are under consideration (e.g., for purchase by the enterprise).
Among the “benchmarks” that an enterprise may wish to consider in evaluating a particular computer system, or comparing a plurality of such systems, is the processing capacity of each system. “Capacity” may be commonly understood as a measure of the maximal amount of computer processing operations that a particular computer system (comprising one or more processors) can perform in a constant period of time. Capacity measurements have been made historically in dimensions of millions of instructions per second (MIPS), but other units may be chosen for measurement of capacity as well. Capacity measurements may be regarded as important for evaluating the practicality (and required system size and configuration) for scaling of server or other computer operations, for side-by-side comparison of multiple systems, for cost analyses, or for other related user evaluations of the computational volume that may be handled by one system vis a vis another.
Capacity does not necessarily dictate that a system of given capacity will actually perform its maximal number of processing operations as used in conjunction with actual programming applications, and does not necessarily indicate whether a system will have desirable operational or output traits as so used with an application. Determining the performance and output of a system as thus actually implemented for applications has posed a separate task from determining capacity. “Performance” may be commonly understood as relating to the quantity of application-specific output (or units of application throughput work) that can be performed by a system in a given unit of time. There is not believed to be, though, any universal definition for what a “unit of work” is in a server environment (although such “unit of work” definitions have been proposed in a mainframe environment). Performance measurement metrics have typically been devised as to each application program or class of application programs.
As the cost of computer systems may well be large, users or potential users will likely wish to learn as much about a particular systems operational traits as possible before, e.g., investing in such system, so as to avoid a poor economic decision in allocating enterprise resources. However, the process of evaluating numerous possible computer systems also places demands upon enterprise resources. At a minimum, employee time must be devoted toward the information gathering and evaluation processes. When a large number of divergent systems are under evaluation, each having a possibly-large number of varying component traits in combination, the process of merely gathering such information may pose a considerable expense of time and money—especially in the case of candidate systems as to which certain system parameters are not readily available (as may occur in the case of a candidate system that consists in part of an already-installed, secondary-market, or other legacy component for which full specifications are not readily available).
Further, existing computer system performance ratings, such as the commonly-known TPC-C, TPC-D, and Specmark ratings, are typically based on running a battery of utilization processes upon an actual candidate server to determine its performance level as actually implemented. In this connection, the Transaction Processing Performance Council (hereinafter TPC) publishes the industry standard performance ratings TPC-C, version 5.0 and TPC-D, version 2.1, which can be found at www.tpc.org/tpcc/spec/tpcc_current.pdf, and www.tpc.org/tpcc/spec/tpcc_current.pdf, respectively, and are attached as Appendix A and B, respectively. The Standard Performance Evaluation Corporation (hereinafter Specmark) publishes a group of benchmark ratings which are maintained and can be found at www.spec.org. Each of these known computer system performance rating standards requires the actual physical testing of processing traits of a specific computer system under actually-implemented testing operations. Configuring multiple candidate systems for actual implementation of testing protocols, as well as gathering the output of such protocols as implemented on the test machines, may impose considerable expense in terms of time, logistics, money, and verification of the accuracy of each empirical test of the physically-configured system.
Thus, it would be desirable to provide a system and method for gaining evaluative information regarding computer system capacity and/or performance by analysis of component combinations, for a broad range of respective actual or theoretical computer systems, without actually running performance tests on a physically-configured example of each candidate system. It would further be desirable to provide such systems and methods requiring only a comparatively-small number of component-characteristic inputs for the analysis, such that extensive information about candidate systems need not be gathered. It would further be desirable to configure such systems and methods in a fashion that provided system metrics consistent with the empirical results of actually-implemented physical testing of actually-configured systems, such that the computational analysis of the analytic, non-empirical method reproduced accurately the empirical data points of as-implemented empirical testing. The prior art is not believed to meet these needs.